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A SYSTEM AND METHOD FOR INTELLIGENT TRANSCODING 



BACKGROUND OF THE INVENTION 

Field of the Invention 

[0001] The present invention is generally related to transcoding. More 
particularly, the present invention is related to a system and method for intelligently 
transcoding video and audio streams to support rendering devices. 

Description 

[0002] Tools exist today that convert from one media format to another, such as, 
for example, Audio Video Interleave (AVI) to Motion Picture Expert Group (MPEG). The 
tools that exist today only do the conversion. They do not take into consideration 
bandwidth requirements, network usage, and/or what type of media is supported by the 
rendering device. In other words, they do not utilize the available resources in a given, 
subsystem effectively. 

[0003] Thus, what is needed is a system and method for converting from one 
media format to another that takes into consideration bandwidth requirements, network 
usage, and the type of media that is supported by the rendering device. What is also 
needed is a system and method for converting from one media format to another that 
effectively utilizes the available resources in the subsystem in which it is delivered. 
What is further needed is a system and method for converting from one media format to 
another without user intervention. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
[0004] The accompanying drawings, which are incorporated herein and form part 
of the specification, illustrate embodiments of the present invention and, together with 
the description, further serve to explain the principles of the invention and to enable a 
person skilled in the pertinent art(s) to make and use the invention. In the drawings, like 
reference numbers generally indicate identical, functionally similar, and/or structurally 
similar elements. The drawing in which an element first appears is indicated by the 
leftmost digit(s) in the corresponding reference number. 

[0005] FIG. 1 is a diagram illustrating an exemplary home network in which the 
present invention may be implemented according to an embodiment of the present 
invention. 

[0006] FIG. 2 is a diagram illustrating the factors involved for providing intelligent, 
transcoding of video and audio streams from one format to another according to an 
embodiment of the present invention. 

[0007] FIG. 3 is a block diagram of an exemplary intelligent transcoding engine 
according to an embodiment of the present invention. 

[0008] FIG. 4A is a flow diagram illustrating an exemplary method for intelligent 
transcoding of video and audio streams from one format to another according to an 
embodiment of the present invention. 

[0009] FIG. 4B is a flow diagram illustrating an exemplary method for determining 
whether intelligent transcoding of video and audio streams may be performed according 
to an embodiment of the present invention. 
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[0010] FIG. 5 is a block diagram illustrating an exemplary computer system in 
which certain aspects of the invention may be implemented. 

DETAILED DESCRIPTION OF THE INVENTION 
[0011] While the present invention is described herein with reference to 
illustrative embodiments for particular applications, it should be understood that the 
invention is not limited thereto. Those skilled in the relevant art(s) with access to the 
teachings provided herein will recognize additional modifications, applications, and 
embodiments within the scope thereof and additional fields in which embodiments of the 
present invention would be of significant utility. 

[0012] Reference in the specification to "one embodiment", "an embodiment" or 
"another embodiment" of the present invention means that a particular feature, structure 
or characteristic described in connection with the embodiment is included in at least one 
embodiment of the present invention. Thus, the appearances of the phrase "in one 
embodiment" appearing in various places throughout the specification are not 
necessarily all referring to the same embodiment. 

[0013] Embodiments of the present invention are directed to a system and 
method for providing intelligent transcoding of video and audio streams from a first data 
format to a second data format. The second data format is supported by the rendering 
device in which the video and/or audio streams are to be played. Intelligent transcoding 
includes, but is not limited to, decoding, encoding, resolution, and bit rate. Intelligent 
transcoding occurs without user intervention. 
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[0014] Embodiments of the present invention are described as being 
implemented in an extended wireless PC (personal computer) home environment. An 
extended wireless PC home environment refers to a home network environment in 
which a PC is used to extend digital media and information access throughout the home 
using wireless technology. Although embodiments of the present invention are 
described using a PC to extend digital media and information access throughout the 
home, one skilled in the relevant art(s) would know that embodiments of the present 
invention may also be implemented in a home or a business environment that 
incorporates other types of computing devices, such as, but not limited to, a media 
center, a set top box, a home server, a workstation, etc., to extend digital media and 
information access throughout the home or business using both wired and wireless 
technology. 

[0015] FIG. 1 is a diagram illustrating an exemplary home network system in 
which the present invention may be implemented according to an embodiment of the 
present invention. Home network system 100 may include a home network 102 that 
receives multimedia content over a wide area network (WAN) 112 from content 
providers, such as, but not limited to, independent content providers 104, broadcast 
operations centers 106, and studios 108. In one embodiment, independent content 
providers 104 and studios 108 may send content to broadcast operations centers 106 to 
format the content and send the content to home network 102. In one embodiment, 
content may come directly from independent content providers 104, broadcast 
operations centers 106, and studios 108 to home network 102. The content may be 
provided to home network 102 using one or more systems 110, such as, but not limited 
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to, a cable system using a coax cable connection, a satellite system using a satellite 
connection, an Internet service using a dial-in connection, a digital telephone connection 
such as DSL (Digital Subscriber Line), a high speed cable modem connection, etc., and 
an ATSC (Advanced Television Systems Committee) system using a high definition 
(HD) connection. The ATSC/Cable/Satellite/lnternet systems 110 obtain information 
from independent content providers 104, broadcast operations centers 106, and studios 
108 to enable audio/video information to be transferred to home network 102. 
[0016] Home network 102 uses a PC 114 to extend digital multimedia content 
and information received from independent content providers 104, broadcast operations 
centers 106, and studios 108 throughout the home using wired and/or wireless 
technology. Although a PC is used to extend digital multimedia content and information, 
other types of computing devices may also be used, such as, but not limited to, a media, 
center, a set-top box, a workstation, a home server, etc. Home network 102 may be 
coupled to WAN 112 via a connection (not shown), such as, a dial-in connection, a high 
speed cable modem connection, a digital subscriber line (DSL) connection, a satellite 
connection, a HD connection, and/or any other means capable of connecting home 
network 102 to WAN 112. 

[0017] Home network 102 includes media Tenderers 122 and 126, and a plurality 
of rendering devices 124 and 128. Media Tenderers 122 and 126 enable an electrical 
connection between devices not ordinarily intended for use together. For example, 
media Tenderer 122 electrically connects PC 114 to rendering devices 124. Media 
Tenderer 126 electrically connects PC 1 14 to rendering devices 128. Rendering devices 
124 and 128 utilize media Tenderers 122 and 126, respectively, in order to receive 



42390.P16772 



-6- 



audio/video input. Rendering devices 124 may include, but are not limited to, a 
personal digital assistant (PDA) 124-1, a television 124-2, and a stereo system 124-3, 
all of which are well known in the relevant art(s). Rendering devices 128 may include, 
but are not limited to, a personal digital assistant (PDA) 128-1 and a television 128-2. 
[0018] In one embodiment, PDA 124-1 and PDA 128-1 may include wireless 
connections, such as, but not limited to, Bluetooth. In this embodiment, PDA 124-1 and 
PDA 128-1 may electrically connect to PC 114 via a wireless connection, thus, 
eliminating the need to connect PDA 124-1 to PC 114 through media Tenderer 122 and 
PDA 128-1 to PC 1 14 through media renderer 126. 

[0019] In one embodiment, PC 1 14 may also receive digital multimedia data from 
other digital devices, such as, but not limited to, an MP3 player 1 16, a digital camcorder 
118, and a digital camera 120. The digital multimedia data received from these digital 
devices may be rendered on one or more of rendering devices 124-1, 124-2, and 124-3 
via PC 114. 

[0020] In one embodiment, MP3 player 116, digital camcorder 118, and digital 
camera 120 may act as rendering devices and/or storage devices. Multimedia content 
from independent content providers 104, broadcast operations centers 106, and studios 
108, may be streamed to any one of devices 116, 118, and 120 for storing and/or 
rendering the media content. 

[0021] As previously indicated, embodiments of the present invention provide a 
method for intelligently transcoding video and audio streams from a first data format to a 
second data format. The second data format is supported by the rendering device on 
which the audio and/or video streams are to be played. Video and audio come in many 
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different formats. Different service providers and different manufacturers of rendering 
devices may provide their content in many different formats, such as, for example, 
MPEG-1 (Motion Pictures Expert Group - 1), MPEG-2 (Motion Pictures Expert Group - 
2), AVI (Audio/Video Interleave), MPEG-4 (Motion Pictures Expert Group - 4), Program 
Stream, Transport Stream (for MPEG AN (AudioA/ideo)), DV (Digital Video), DivX, Real 
AA/ (Real AudioA/ideo), WMV/WMA (Windows Media Video/Windows Media Audio 
developed by Microsoft Corporation), etc. These are just a few of the media formats 
available. All rendering devices do not support all of the media formats; hence, there is 
a need to convert media from one format to another format to enable interoperability 
across media devices. New codecs evolve at a very fast rate than their penetration into 
the hardware world. It therefore becomes almost impossible to achieve interoperability 
if conversion from one media format to another media format is not enabled. 
[0022] When a user of a rendering device, such as one of rendering devices 124- 
1, 124-2, 124-3, 128-1, or 128-2 wants to play a particular media selection in the home 
network environment, such as in home network 102, intelligent transcoding determines 
the supported media format(s) that the selected rendering device supports. For 
example, using UPnP (Universal Plug and Play) control points and discovery methods, 
information regarding rendering device capabilities may be obtained. UPnP is well 
known to those skilled in the relevant art(s). One skilled in the relevant art(s) would also 
know that other methods for obtaining information about the capabilities of rendering 
device are available, such as, for example, using a metadata server to discover 
rendering device capabilities. Intelligent transcoding analyzes network properties, along 
with supported media types on the rendering device, and decides which format to 
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transcode the media content into for playing on the rendering device. Accordingly, 
intelligent transcoding will transcode the media content and broadcast or stream it in the 
given network environment to the appropriate rendering device. 
[0023] In a home network environment, networked devices have limited rendering 
and decoding capabilities. The home network also has limited resources, such as 
limited dynamic memory, processing load, and available network bandwidth. Intelligent 
transcoding considers (1) the type of media that is supported by both the server devices 
and the rendering devices; and (2) network capabilities, such as network bandwidth 
requirements, processor load, and available memory, to determine if transcoding is 
possible and, if it is possible, transcodes the media in a proficient manner. 
[0024] FIG. 2 is a diagram 200 illustrating the components involved in performing 
intelligent transcoding according to an embodiment of the present invention. Intelligent 
transcoding requires the ability to perform transrating, transcaling, transformatting, 
transcripting, and transcoding. Transrating is the process of changing or reducing the 
bitrate of the source content. Transcaling is the process of changing the resolution of 
the source content. Transformatting is the process of converting the packaging of the. 
media format to another packaging format. Transcripting is the process of converting 
from one Digital Rights Management (DRM)/copy protection scheme to another 
DRM/copy protection scheme. Transcoding is the process of converting one media 
format into another media format. The components involved in performing intelligent 
transcoding include a rules based engine 202 (also referred to as policy manager 202), 
a network throughput engine 204, and a platform usage engine 206. Rules based 
engine 202, network throughput engine 204, and platform usage engine 206 each 
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provide information to a transcoding engine 208. The information provided enables 
transcoding engine 208 to perform intelligent transcoding. In other words, transcoding 
engine uses the information from rules based engine 202, network throughput engine 
204, and platform usage engine 206 to determine whether transrating, transcoding, 
and/or transcaling may be performed. 

[0025] Rules based engine/policy manager 202 defines the rules, which 
incorporate policy based principles, which are applied to determine whether intelligent 
transcoding can be performed. Rendering devices have different rendering capabilities. 
To account for this, rules based engine/policy manager 202 defines the applicable 
media formats in which a particular media format may be transcoded. Rules based 
engine/policy manager 202 also determines the required platform usage for a particular 
format conversion. 

[0026] In one embodiment, the rules that are used in rules based engine/policy 
manager 202 are implemented in XML (extensible markup language). Implementing the 
rules in XML provides an operator the ability to modify the rules with little effort. 
[0027] Network throughput is a measure, in bits per second, of the traffic carrying 
capacity of the network. Network throughput engine 204 determines network bandwidth 
availability. By knowing the available throughput of the network, the transcoding bit rate 
may be adjusted dynamically during intelligent transcoding. For example, if the source 
content format is a MPEG-2 transport stream with a bit rate of 6 Mbps and the rendering 
device supports MPEG-2 transport stream, but the network availability is 3 Mbps, the 
network cannot support the source content. In this instance, transcoding engine 208 
does not have to transcode the source content because both the source and the 
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rendering device support MPEG-2 transport stream. Instead, transcoding engine 208 
has to perform transrating to lower the bit rate of the source content. In other words, 
transcoding engine 208 needs to drop the bit rate of the source content from 6 Mbps 
down to 3 Mbps to enable the source content to be streamed to the rendering device. 
[0028] Thus, network throughput engine 204 will determine the available bit rate 
or bandwidth on the network and feed that information back into transcoding engine 
208. Transcoding engine 208 will analyze the information from network throughput 
engine 206, along with the input from policy manager 202, to make decisions as to 
whether or not there is enough network throughput to send the data to the rendering 
device. 

[0029] Platform usage engine 206 determines the current load on the processor, 
how much processor power is currently available, how much memory is available, and 
whether intelligent transcoding can be done given such processor and memory usage. 
For example, if content is to be streamed on two devices, such as device 124-1 and 
device 128-2, but the processor does not have the capability to transcode media to both 
devices 124-1 and 128-2, then platform usage engine 206 will provide transcoding 
engine 208 with the available platform usage, and transcoding engine 208 will 
determine which rendering device, if any, can be accommodated. 
[0030] Thus, the information obtained from rules based engine/policy manager 
202, network throughput 204, and platform usage 206 each contribute to intelligently 
transcoding digital media and are input data for transcoding engine 208. Transcoding 
engine 208 will, in turn, decide which format to convert to, what the bitrate format should 
be, whether the resolution needs to be altered (e.g., high definition (HD) to standard 
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definition (SD)), whether the packaging of the media format needs to be changed, 
whether the DRM/copy protection scheme needs to be changed, and whether the 
network has the capacity to perform the transcoding. If transcoding engine 208 decides 
that the capability is available, transcoding engine 208 will intelligently transcode the 
data and stream the data to the rendering device, such as, for example, rendering 
device 124-1, 124-2, or 124-3, directly or via digital media renderer 122. 
[0031] FIG. 3 is a block diagram 208 illustrating an exemplary intelligent 
transcoding engine according to an embodiment of the present invention. Transcoding 
engine 208 may be implemented in software, hardware, or a combination thereof. 
Transcoding engine 208 comprises policy manager 202, a transport manager 302, a 
back channel manager (BCM) 304, a BCM network protocol 306, a graph manager 308, 
a HTTP (Hypertext Transport Protocol) server 320, and a RTP (Real Time Transport 
Protocol) server 322. 

[0032] As previously indicated, policy manager 202 includes rules that define the 
applicable media formats in which a particular media format may be transcoded and 
determines the required platform usage for a particular format conversion. Transport 
manager 302 is responsible for communicating with an application layer (not shown), 
such as, for example, UPnP for determining device characteristics. Back channel 
manager (BCM) 304 is responsible for communicating out of band communications or 
commands. For example, commands not supported by UPnP, such as, for example,, 
autostop notifications and trick mode commands (e.g., fast forward, rewind, seek), may 
be handled by BCM 304. BCM network protocol 306 is used to provide the appropriate 
protocol to enable BCM to handle out of band communications or commands. 
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[0033] Transport manager 302 gathers the information from policy manager 202, 
network throughput 204, and platform usage 206 and communicates this information to 
graph manager 308. Graph manager 308 then puts together a graph or infrastructure 
for intelligent transcoding. Graph manager 308 includes a source/capture filter 310, a 
demultiplexer 312, a decode/encode 314, a multiplexer 316, and a network filter 318. 
Intelligent transcoding is performed using source/capture filter 310, demultiplexer 312, 
decode/encode 314, multiplexer 316, and network filter 318. Source/capture filter 310 
receives media data 324 as input and filters the media data. Demultiplexer 312 
separates the media data into video and audio components. Decode/Encode 314 
decodes the media format and intelligently transcodes the media format based on the 
infrastructure designated by graph manager 308. Again, intelligent transcoding includes 
decoding, encoding, transrating, transformatting, transcripting, and transcaling. In the 
case of decoding, in an embodiment, a full decode to raw video bits may be performed 
or decoding may be performed to a degree where commonality between the streams 
can be used to partially decode and re-encode from that point. Multiplexer 316 
combines the intelligently transcoded video and audio together. Network filter 318 filters 
the media signal. HTTP server 320 and RTP server 322 put the filtered media data 
onto the network for streaming to the rendering devices, such as, for example rendering 
devices 124-1, 124-2, and 124-3. HTTP server 320 is a pull model for when clients 
request data. RTP server 322 is a push model for enabling the server to push data onto 
the client side. 

[0034] FIG. 4A is a flow diagram 400 illustrating a method for intelligent 
transcoding of video and audio streams from one format to another according to an 
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embodiment of the present invention. The invention is not limited to the embodiment 
described herein with respect to flow diagram 400. Rather, it will be apparent to 
persons skilled in the relevant art(s) after reading the teachings provided herein that 
other functional flow diagrams are within the scope of the invention. The process 
begins with block 402, where the process immediately proceeds to block 404. 
[0035] In block 404, a user is enabled to select a media item that the user desires 
to have played on a particular rendering device. A request for the media item to be 
played is made to the server side in block 406. In block 408, the media item is received 
from the server side. The process then proceeds to decision block 410. 
[0036] In decision block 410, it is determined whether the media item received 
needs intelligent transcoding in order to be played on the rendering device. In order to 
determine whether intelligent transcoding is needed, the device capabilities of the 
rendering device or devices must be determined. As previously indicated, this may be 
accomplished using UPnP control point and discovery methods. Alternatively, other 
methods may be used to determine the device capabilities, such as, but not limited to, 
using a metadata server to discover rendering device capabilities. If the media item 
received needs intelligent transcoding in order to be played on the rendering device, the 
process proceeds to decision block 412. 

[0037] In decision block 412, it is determined whether intelligent transcoding of 
the media item may be performed. 

[0038] FIG. 4B is a flow diagram 412 describing a method for determining 
whether intelligent transcoding may be performed. The invention is not limited to the 
embodiment described herein with respect to flow diagram 412. Rather, it will be 
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apparent to persons skilled in the relevant art(s) after reading the teachings provided 
herein that other functional flow diagrams are within the scope of the invention. The 
process begins with block 414, where the rules in policy manager 202 are examined to 
determine whether intelligent transcoding may be performed. If the rules allow for 
transcoding, policy manager 202 will then provide the required platform usage for 
transcoding. The process then proceeds to block 416. 

[0039] In block 416, it is determined whether the required platform usage to 
perform the transcoding is available. As previously indicated, the platform usage looks 
to available processor power and memory to determine whether there is the capacity to 
perform the transcoding. The process then proceeds to block 418. 
[0040] In block 418, network throughput is examined to determine whether there 
is enough bandwidth in the network to perform the transcoding. The process then 
proceeds to decision block 420. 

[0041] In decision block 420, it is determined whether intelligent transcoding can 
be performed given the rules, the required platform usage, the platform capacity 
available, and network throughput. If intelligent transcoding can be performed, the 
process proceeds to block 422 in FIG. 4A. If intelligent transcoding cannot be 
performed, the process proceeds to block 428 in FIG. 4A. 

[0042] Returning to FIG. 4A, block 412, if it is determined that intelligent 
transcoding of the media item may be performed, the process then proceeds to block 
422. 

[0043] In block 422, the media content is input to transcoding engine 208 for 
performing one or more of transrating, transcaling, transformatting, transcripting, and 
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transcoding. In block 424, the transcoded media content is streamed to the rendering 
device. The process then proceeds to block 428, where the process ends. 
[0044] Returning to decision block 412, if it is determined that intelligent 
transcoding may not be performed, the process proceeds to block 428, where the 
process ends. 

[0045] Returning to decision block 410, if it is determined that the media content 
received from the server side does not need intelligent transcoding, then the process 
proceeds to block 426. In block 426, the media content is streamed to the appropriate 
rendering device. The process then proceeds to block 428, where the process ends. 
[0046] Embodiments of the present invention may be implemented using 
hardware, software, or a combination thereof and may be implemented in one or more 
computer systems or other processing systems. In fact, in one embodiment, the 
invention is directed toward one or more computer systems capable of carrying out the 
functionality described here. An example implementation of a computer system 500 is 
shown in FIG. 5. Various embodiments are described in terms of this exemplary 
computer system 500. After reading this description, it will be apparent to a person 
skilled in the relevant art how to implement the invention using other computer systems 
and/or computer architectures. 

[0047] Computer system 500 includes one or more processors, such as 
processor 503. Processor 503 is connected to a communication bus 502. Computer 
system 500 also includes a main memory 505, preferably random access memory 
(RAM) or a derivative thereof (such as SRAM, DRAM, etc.), and may also include a 
secondary memory 510. Secondary memory 510 may include, for example, a hard disk 
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drive 512 and/or a removable storage drive 514, representing a floppy disk drive, a 
magnetic tape drive, an optical disk drive, etc. Removable storage drive 514 reads from 
and/or writes to a removable storage unit 518 in a well-known manner. Removable 
storage unit 518 represents a floppy disk, magnetic tape, optical disk, etc., which is read 
by and written to by removable storage drive 514. As will be appreciated, removable 
storage unit 518 includes a computer usable storage medium having stored therein 
computer software and/or data. 

[0048] In alternative embodiments, secondary memory 510 may include other 
similar means for allowing computer programs or other instructions to be loaded into 
computer system 500. Such means may include, for example, a removable storage unit 
522 and an interface 520. Examples of such may include a program cartridge and 
cartridge interface (such as that found in video game devices), a removable memory 
chip (such as an EPROM (erasable programmable read-only memory), PROM 
(programmable read-only memory), or FLASH memory) and associated socket, and 
other removable storage units 522 and interfaces 520 which allow software and data to 
be transferred from removable storage unit 522 to computer system 500. 
[0049] Computer system 500 may also include a communications interface 524. 
Communications interface 524 allows software and data to be transferred between 
computer system 500 and external devices. Examples of communications interface 524 
may include a modem, a network interface (such as an Ethernet card), a 
communications port, a PCMCIA (personal computer memory card international 
association) slot and card, a wireless LAN (local area network) interface, etc. Software 
and data transferred via communications interface 524 are in the form of signals 528 
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which may be electronic, electromagnetic, optical or other signals capable of being 
received by communications interface 524. These signals 528 are provided to 
communications interface 524 via a communications path (i.e., channel) 526. Channel 
526 carries signals 528 and may be implemented using wire or cable, fiber optics, a 
phone line, a cellular phone link, a wireless link, and other communications channels. 
[0050] In this document, the term "computer program product" refers to 
removable storage units 518, 522, and signals 528. These computer program products 
are means for providing software to computer system 500. Embodiments of the 
invention are directed to such computer program products. 

[0051] Computer programs (also called computer control logic) are stored in main 
memory 505, and/or secondary memory 510 and/or in computer program products. 
Computer programs may also be received via communications interface 524. Such 
computer programs, when executed, enable computer system 500 to perform the 
features of the present invention as discussed herein. In particular, the computer 
programs, when executed, enable processor 503 to perform the features of 
embodiments of the present invention. Accordingly, such computer programs represent 
controllers of computer system 500. 

[0052] In an embodiment where the invention is implemented using software, the 
software may be stored in a computer program product and loaded into computer 
system 500 using removable storage drive 514, hard drive 512 or communications 
interface 524. The control logic (software), when executed by processor 503, causes 
processor 503 to perform the functions of the invention as described herein. 
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[0053] In another embodiment, the invention is implemented primarily in 
hardware using, for example, hardware components such as application specific, 
integrated circuits (ASICs). Implementation of hardware state machine(s) so as to 
perform the functions described herein will be apparent to persons skilled in the relevant 
art(s). In yet another embodiment, the invention is implemented using a combination of 
both hardware and software. 

[0054] While various embodiments of the present invention have been described 
above, it should be understood that they have been presented by way of example only, 
and not limitation. It will be understood by those skilled in the art that various changes 
in form and details may be made therein without departing from the spirit and scope of 
the invention as defined in the appended claims. Thus, the breadth and scope of the 
present invention should not be limited by any of the above-described exemplary 
embodiments, but should be defined in accordance with the following claims and their 
equivalents. 
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